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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )M Responsive to communication(s) filed on 4/29/05 & 6/16/05 . 
2a)I3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 1-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [3 Claim(s) 1-19 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)H The drawing(s) filed on 26 April 2005 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)DAII b)d Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Detailed Action 
Response to Amendment 

1 . The amendment filed on 4/29/2005 and 6/1 6/2005 have been entered. 
The objection to the title is overcome by applicant's amended title. 

Response to Arguments 

2. The objection to figures 7 and 1 1 are withdrawn in view of applicants arguments 
filed on 4/29/2005. 

3. The arguments filed on 4/29/2005 concerning claim 10 and the Kitamura article 
have been fully considered, however, they are deemed not to be persuasive. Claim 10 
does not have an image taking step and thus does not claim the obtaining a 
constraining shape is separate from the image taking step. Therefore Kitamura teaches 
this broad method. 

4. The arguments concerning the 35 USC 112 second paragraph rejection of claims 
9 and 15 is not persuasive. Claim 9 depends upon claim 8 which claims A computer 
program, wherein the information processing method according to Claim 5 is executed 
by a computer device. A connection between the computer program and the wherein 
clause does not exist, thus, a computer program has not been claimed, just the words 
"A computer program" is claimed. The same argument applies to claim 15. Applicant 
needs to amend claims 8, 9, 14, and 15 to better claim a computer-readable recording 
medium embodying computer instructions causing a computer to perform the claimed 
method. Claims 8 and 14 claim a program per se and cannot be allowed as such. 
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Claims 9 and 15 do not claim what type of computer program is stored, thus, they are 
non functional descriptive material. 



Claim Rejections - 35 USC §112 

5. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claims 1-9 and 16-19 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 

Claim 1: 

At line 4 "in real space" , at line 5 "on real space" and at line 10 "in real space" is 
claimed, however, the claim does not clearly the claim the third occurrence of "real 
space" is a new claim limitation or is further limiting the first occurrence of "real space". 
The examiner suggests changing at line 5 "on real space" to "on said real space" and 
changing at line 10 "in real space" to "in said real space" to make the claim clearly claim 
the user is inputting 3D position in the real space of the image taken by the image 
taking means. 

At line 3 "three-dimensional space" and at lines 9 and 13 "three-dimensional 
position information" is claimed and as discussed above "real space" is claimed at lines 
4, 5, and 10. A correlation between "three-dimensional space", "three-dimensional 
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position information" and "real space" is needed to clearly claim controlling the position 
and orientation of the virtual object. 

At lines 16-17 "the user's instructions is claimed and lacks antecedent basis in 
the claims. 



Claims 3 and 18: 

These dependent claims were amended to claim "at least one of the following 

operations... translation.... and ...rotation". In view of Superguide Corp. v. DirecTV 

Enterprises, Inc. and IPXL Holdings LLC v. Amazon.com Inc. this claim does not clearly 

claim the described invention since this phrase is inclusive rather than alternative. The 

specification did not specifically state the number of times both translation and rotation 

occurs. The specification describes with regard to step 8010 correcting the position and 

orientation of the virtual object. Superguide Corp. v. DirecTV Enterprises, Inc., 358 

F.3d 870, 69 USPQ2d 1865, 1878 (Fed. Cir. 2004). Page 1878 states: 

We agree with DirecTV. The phrase "at least one of precedes a series of categories of 
criteria, and the patentee used the term "and" to separate the categories of criteria, 
which connotes a conjunctive list. A common treatise on grammar teaches that "an 
article of a preposition applying to all the members of the series must either be used 
only before the first term or else be repeated before each term." Willaim Strunk, Jr. & E. 
B. White, The Elements of Style 27 (4th ed. 2000). Thus, "[i]n spring, summer, or winter" 
means "in spring, in summer, or in winter." Id. Applying this grammatical principle here, 
the phrase "at least one of modifies each member of the list, i.e., each category in the 
list. Therefore, the district court correctly interpreted this phrase as requiring that the 
user select at least one value for each category; that is, at least one of a desired 
program start time, a desired program end time, a desired program service, and a 
desired program type. 
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IPXL Holdings LLC v. Amazon.com Inc., 72 USPQ2d 1469, 1480 (DC EVa 2004) 
makes a decision similar to SuperGuide. Page 1480 states: 

The parties dispute whether "user defined transaction information" requires both a "user 
defined transaction" and a "user defined transaction parameter" as Amazon contends, 
or only one of either a "user defined transaction" or a "user defined transaction 
parameter," as IPXL contends. For the reasons set forth below, the Court finds that 
IPXL misreads both the plain words used in the claim as well as Federal Circuit 
precedent in arguing for its interpretation. In construing the very same language, the 
Federal Circuit construed "at least one of to mean what Amazon has argued. 
The phrase "at least one of" precedes a series of categories of criteria, and the 
patentee used the term "and"to separate the categories of criteria, which connotes a 
conjunctive list. A common treatise on grammar teaches that "an article of a preposition 
applying to all the members of the series must either be used only before the first term 
or else be repeated before each term." William Strunk, Jr. & E.B. White, The Elements 
of Style 27 (4th ed. 2000)... . Applying this grammatical principle here, the phrase "at 
least one of "modifies each member of the list, i.e., each category in the list. Therefore, 
the district court correctly interpreted this phrase as requiring that the user select at 
least one value for each category... .SuperGuide Corp. v. DirecTV Enterprises, Inc., 358 
F.3d 870, 886 f69 USPQ2d 18651 (Fed. Cir. 2004). 



CAFC decision Brown v. 3M, 265 F.3d 1349, 60 USPQ2d 1375 (Fed. Cir. 2001) 
This decision found that the term "or" in claim 16 in the at least one of two-digit, three- 
digit, or four-digit year-date representations phrase is to be read in the alternative when 
read in light of the specification. Page 1378 states: 

The district court construed the word "or"in claim 16 as meaning that the apparatus was 
capable of converting "only two-digit, only three-digit, only four-digit, or any combination 
of two-, three-, and four-digit date-data." Slip op. at 9. We agree with this construction 
of the claim, for it is the plain reading of the claim text. These are not technical terms of 
art, and do not require elaborate interpretation. There is no basis in the specification or 
prosecution history for reading "or"as "and" — nor does Dr. Brown request such a 
reading. 
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Claims 4 and 19: 

These claims were amended to claim "real time" however, the specification does 
not use the phrase "real time" and paragraph 0036 appears to suggest "real time" but 
not "real time in accordance with the user's instructions". 

Claim 5: 

This claim has the same indefinite issues that claim 1 has. 

At line 4 "in real space" , at line 6 "on real space" and at line 9 "in real space" is 
claimed, however, the claim does not clearly the claim the third occurrence of "real 
space" is a new claim limitation or is further limiting the first occurrence of "real space". 
The examiner suggests changing at line 6 "on real space" to "on said real space" and 
changing at line 9 "in real space" to "in said real space" to make the claim clearly claim 
the user is inputting 3D position in the real space of the image taken by the image 
taking step. 

At lines 2- 3 "three-dimensional space" and at lines 8 and 13 "three-dimensional 
position information" is claimed and as discussed above "real space" is claimed at lines 
4, 6, and 9. A correlation between "three-dimensional space", "three-dimensional 
position information" and "real space" is needed to clearly claim controlling the position 
and orientation of the virtual object. 

At line 17 "the user's instructions is claimed and lacks antecedent basis in the 
claims. 
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Claim 16: 

This claim has the same indefinite issues that claim 1 has. 

At line 4 "in real space" , at line 5 "on real space" and at line 9 "in real space" is 
claimed, however, the claim does not clearly the claim the third occurrence of "real 
space" is a new claim limitation or is further limiting the first occurrence of "real space". 
The examiner suggests changing at line 5 "on real space" to "on said real space" and 
changing at line 9 "in real space" to "in said real space" to make the claim clearly claim 
the user is inputting 3D position in the real space of the image taken by the image 
taking step. 

At line 3 "three-dimensional space" and at lines 8 and 1 1 "three-dimensional 
position information" is claimed and as discussed above "real space" is claimed at lines 
4, 5, and 9. A correlation between "three-dimensional space", "three-dimensional 
position information" and "real space" is needed to clearly claim controlling the position 
and orientation of the virtual object. 

At line 13 "the user's instructions is claimed and lacks antecedent basis in the 
claims. 

Claim 9: 

Claim 9 is indefinite because "the computer program" lacks antecedent basis in 
the claim. Claim 8 does not claim a computer program, it just has the words computer 
program because the wherein clause is not connected to the computer program. 
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Claim 15: 

Claim 15 is indefinite because "the computer program" lacks antecedent basis in 
the claim. Claim 14 does not claim a computer program, it just has the words computer 
program because the wherein clause is not connected to the computer program. 

Claim Objections 

7. Claims 8, 9, 14, and 15 are objected to because of the following informalities: 
The form of these claims are objected as being programs per se rather than computer- 
readable recording medium embodying computer instructions causing a computer to 
perform the claimed method. Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

9. Claims 10-15 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Yoshifumi Kitamura and Fumio Kishino, Consolidated Manipulation of Virtual and Real 
Objects, September 1997, Proceedings of the ACM symposium on Virtual reality 
software and technology, pages 133-138. Kitamura teaches an augmented reality 
system that uses object constraints to control the visual interaction between the virtual 
objects and the real objects. Claim 10 does not have an image taking step and thus 
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does not claim the obtaining a constraining shape is separate from the image taking 
step. Therefore Kitamura teaches this broad method. 
A detailed analysis of the claims follows. 

Claim 10: 

Kitamura teaches an information processing method for changing the position 
and orientation of a virtual object in mixed reality space obtained by combining a real 
image and a virtual image, said method comprising the steps of: 

obtaining a constraining shape from a plurality of positions in real space 
designated by a user using an operating unit capable of obtaining three-dimensional 
positional information (The three paragraphs found in section 2 on page 134 of 
Kitamura teaches the user measuring the real objects. Kitamura discusses in section 2 
in the first paragraph using conventional modeling software after precisely measuring the 
size or length of the real object by hand which is an operating unit capable of obtaining 
three-dimensional positional information . ); 

changing the position and orientation of the virtual object according to 
instructions from the user, based on the obtained constraining shape as constraint 
conditions (The first paragraph in section 5 on page 135 describes the user using a 6 
DOF tracker device to manipulate the virtual objects. The shape of the real objects are 
used to constrain the movement of the virtual object by giving the real object a shape 
that the virtual object interacts with in a constrained manner. Sections 5. 1 to 5.4 
discusses manipulation of the virtual object based upon constraint conditions based on 
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the shape of the real object. The shape generated from the external instructions 
constraints the interaction of the virtual object with the real object, see sections 5. 1 to 
5.4.); and 

combining an image of the virtual object generated according to the changed 
position and orientation, and the real image {The introduction on page 133 second full 
paragraph discusses augmented reality which synthesizes a virtual object with a real object. 
The claim does not limit the form of the real image, thus, Kitamura teaches this limitation. 
Section 5. 1 discusses after movement is detected by the 6DOF manipulator the virtual object 
is moved according to the constraints.). 

Claim 11: 

Kitamura teaches an information processing method according to Claim 10, 
further comprising the step of combining a virtual image indicating the constraining 
shape with the real image (The introduction on page 1 33 second full paragraph discusses 
augmented reality which synthesizes a virtual object with a real object. The claim does not 
limit the form of the real image, thus, Kitamura teaches this limitation. Section 5. 1 discusses 
after movement is detected by the 6DOF manipulator the virtual object is moved according to 
the constraints.). 

Claim 12: 

Kitamura teaches an information processing method according to Claim 10, 
wherein the constraining shape is a plane. (Onpage 136 in the text above figure 2 determining a 
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plane and using the plane to constrain movement of the virtual object is discussed with regards to figure 
2 > 

Claim 13: 

Kitamura teaches an information processing method according to Claim 10, 
wherein said changing step changing the position and orientation of the virtual object is 
carried out by changing the position and orientation of the operating unit (The 6 DOF 
tracker is an operating unit. The user using the 6 DOF tracker device manipulates the 
virtual objects.). 

Claim 14: 

Kitamura teaches a computer program, wherein the information processing 
method according to Claim 1 0 is executed by a computer device (This article is directed 
to computers that generate the augmented reality scene since it was published by ACM 
for a symposium on virtual reality software and technology and since at page 133 in the 
last sentence in the second paragraph of section 1 software/hardware is discussed.). 

Claim 15: 

Kitamura teaches a computer-readable recording medium, storing the computer 
program according to Claim 14 (Software causing a computer to perform Kitamura's 
augmented reality is inherently stored in a computer readable recording medium.). 
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Allowable Subject Matter 

10. Claims 1-7, 16-13, and 16-19 would be allowable if rewritten or amended to 
overcome the rejection(s) under 35 U.S.C. 1 12, 2nd paragraph, set forth in this Office 
action. The prior art of record fails to teach or suggest taking a real image, 
superimposing a virtual image with the real image, having the user input positions in 
real space that are used to form the constraining shape. While Kitamura teaches taking 
and analyzing a real image for the constraining shape or having the user input the 
positions for the constraining shape. 

1 1 . THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1..136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jeffery A Brier whose telephone number is (571 ) 272- 
7656. The examiner can normally be reached on M-F from 7:00 to 3:30. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, Michael 
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Razavi, can be reached at (571) 272-7664.. The fax phone Number for the organization 
where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




Jeffery A Brier 
Primary Examiner 
Art Unit 2672 
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